Papers with Glancing Language Model
Glancing Transformer for Non-Autoregressive Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translation methods are either inferior to Transformer or require multiple decoding passes, leading to reduced speedup. |
| Approach: | They propose a Glancing Language Model (GLM) for single-pass parallel generation models and Glancing Transformer (GLAT) with only single- pass decoding, GLAT is able to generate high-quality translation with 8-15 speedup. |
| Outcome: | The proposed model outperforms all previous non-autoregressive methods on multiple language directions and is nearly comparable to Transformer. |